Data storage servers (“storage servers”) can store data redundantly, e.g., across multiple data storage devices. Storage servers may employ various forms of data storage devices, e.g., hard disk drives, solid state drives, tape devices, etc. The data storage devices are typically implemented as one or more storage “volumes” that comprise a cluster of data storage devices, in which the volumes define an overall logical arrangement of storage space. For example, a storage server can serve a large number (e.g., 150 or more) of discrete volumes. Each volume is generally associated with its own file system (e.g., a write anywhere file system).
To improve performance, storage servers can temporarily store various data storage commands (“storage operations” or simply “operations”) they receive from client computing devices in a region of system memory of the storage servers. By storing the data they receive in system memory, the storage servers can immediately return an acknowledgement message to the client computing devices rather than waiting for slower data storage devices to first actually store the data first. However, system memory can be erased before the data is stored to data storage devices, e.g., in an event of a power (or other) failure.
To reduce the likelihood of data loss in such circumstances, the storage servers may also store the data in nonvolatile random access memory (NVRAM), e.g., in a log file stored in the NVRAM. By initially storing the operations in the NVRAM log file, the storage server can immediately return an acknowledgment to the client computing devices rather than wait for the operation to complete on one or more data storage devices. Moreover, in the event of failure of the storage server or data storage devices, the storage operations can be “replayed,” thereby preventing loss of data. The NVRAM can have various associated circuitry to prevent data loss, e.g., battery backup, FLASH-type memory, etc. By logging storage operations (e.g., create file, write data, delete data, etc.) as “journal” entries in the log file, the storage server can conform with data storage protocols that require the storage server not to acknowledge the storage operation before writing data to persistent storage.
The NVRAM log file can accumulate storage operations until a consistency point (CP) is triggered. CPs can be triggered at various time intervals (e.g., fixed time intervals), or when other events arise, e.g., the NVRAM is almost fully filled. At each CP, data is transferred from the storage server system memory to underlying data storage volumes, and the NVRAM is cleared of the transferred data upon successful transfer.
If the storage server's operations are interrupted unexpectedly, e.g., because of power failure or other subsystem problem, its operating system or file system (e.g., write anywhere file system) can recover information by using information stored in the NVRAM log file between the time of the last CP and the unexpected interruption, e.g., by using a “replay” operation.
Technological advances have caused a significant reduction in the price of NVRAM and processors with a concomitant increase in logic density. Thus, it is now possible to employ much more NVRAM and processors (e.g., processor “cores”) at a lower cost than was previously possible. For example, a storage server can employ multiple gigabytes of NVRAM and 16 or more processor cores. It can be desirable to have a large amount of NVRAM to increase the throughput of the storage server. On the other hand, having more storage operations that are saved in the log file can cause an increase in the time required to complete a playback (or replay operation). To make it possible for the storage server to operate at high speed while maintaining an acceptably small recovery time, it is desirable to reduce the replay time per operation to compensate for the greater number of operations being recorded in the log file.